What is claimed is: 

1. An interactive operation supporting system for an 
apparatus, comprising a display unit, a speech input unit, 
a speech output unit and an operation control unit, wherein 
5 said operation control unit includes: 

an assistant control means for generating a personified 
assistant and making said assistant appear on a screen of said 
display unit; 

an output speech control means for determining speech 
10 required for said assistant to output said assistant's speech 
to the outside through said speech output unit after speech 
synthesis ; 

an input speech recognition means for recognizing user ' s 
voice as a speech inputted through said speech input unit; 

15 an interaction management means for managing interaction 

between said assistant and said user according to said 
assistant's speech determined by said output speech control 
means and said user speech recognized by said input speech 
recognition means; and 

20 a command interpreting means for specifying a user's 

intention or specifying said inputted user command based on 
a content of interaction traced by said interaction management 
means . 

25 2 . The interactive operation supporting system according 
to claim 1, wherein said assistant control means determines 
an animation of said assistant based on a content of interaction 
managed by said interaction management means and/or an inputted 
user command specified by said command interpreting means . 

30 

3 . The interactive operation supporting system according 



67 



to claim 1 , wherein said output speech control means determines 
an assistant ' s speech based on a content of interaction managed 
by said interaction management means and/or an inputted user 
command specified by said command interpreting means - 

5 

4. The interactive operation supporting system according 
to claim 1 , wherein said output speech control means determines 
an assistant ' s speech for leading a user ' s intention when said 
command interpreting means fails to specify said user's 

10 intention or said inputted user command. 

5. The interactive operation supporting system according 
to claim 1 , further comprising a connection means for connecting 
an external apparatus such as a television set/monitor and/or 

15 a video deck to said system, wherein 

said command interpreting means also interprets a 
command for controlling a function of said external apparatus 
including selection of a broadcast program channel and/or 
recording/reproducing of said broadcast program. 

20 

6 . The interactive operation supporting system according 
to claim 1, further comprising a communication means for 
connecting said system to a communication medium such as an 
external network or a telephone line , wherein said input speech 

25 recognition means recognizes audio data received via said 
communication medium. 

7. The interactive operation supporting system according 
to claim 1, further comprising a communication means for 

30 connecting said system to a communication medium such as an 
external network and/or a telephone line, and a mail exchange 
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means for making an exchange of electronic mails via said 
communication medium, wherein said output speech control means 
determines said assistant's speech based on a content of an 
incoming mail. 

5 

8. The interactive operation supporting system according 
to claim 1 , wherein said interaction management means manages 
an input speech of a user as a message bound for another user, 
and said output speech control means determines said 

10 assistant's speech based on said message. 

9 . The interactive operation supporting system according 
to claim 1, wherein said assistant control means places said 
personified assistant in a room scattered with various kinds 

15 of objects including a link to an information resource. 

10. The interactive operation supporting system according 
to claim 1, wherein said assistant control means places said 
personified assistant in a room scattered with various kinds 

20 of objects including a link to an information resource, and 
in response to an interest of said user in a recording medium 
including a link to a music content placed in said room, said 
command interpreting means interprets an inputted user command 
as a command for playing back said music content. 

25 

11. The interactive operation supporting system according 
to claim 1 , wherein in response to said command interpreting 
means interpreting an inputted user command, said assistant 
control means makes said assistant appear on a screen of said 

30 display unit. 
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12. The interactive operation supporting system according 
to claim 1 , further comprising a connection means for connecting 
a television set /monitor to said system, and in response to 
said command interpreting means interpreting an inputted user 

5 command as a channel selection command, said assistant control 
means makes said assistant show a selected broadcasting program 
display window. 

13. The interactive operation supporting system according 
10 to claim 1 , further comprising a connection means for connecting 

a television set/monitor to said system, wherein in response 
to said command interpreting means interpreting an inputted 
user command as a channel change command , said assistant control 
means placing a ring-shaped changeable broadcasting program 
15 display window around said assistant . 

14. The interactive operation supporting system according 
to claim 13, wherein, in response to selection of a desired 
channel by moving said display window on said ring upon a 

20 channel change command from said user and said assistant control 
means also zooms up said selected broadcasting program display 
window . 

15. The interactive operation supporting system according 
25 to claim 1 , further comprising a connection means for connecting 

a secondary storage device for storing and/or reproducing a 
broadcasting program content to said system, wherein in 
response to said command interpreting means interpreting an 
inputted user command as a recorded program reproduction 
30 command, said assistant control means makes said assistant 
show a binder showing a view of recorded broadcasting program 
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contents . 

16. The interactive operation supporting system according 
to claim 15, wherein in response to a selection for reproducing 

5 a recorded broadcasting program content, said assistant 
control means zooms up said selected recorded broadcasting 
program content display window. 

17. The interactive operation supporting system according 
10 to claim 1 , further comprising a connection means for connecting 

a television set/monitor to said system, wherein in response 
to said command interpreting means interpreting an inputted 
user command as a channel change command , said assistant control 
means makes said assistant appear with a list of changeable 
15 broadcasting programs arranged in a matrix shape. 

18. The interactive operation supporting system according 
to claim 17, wherein in response to a selection of a desired 
channel, said assistant control means zooms up said selected 

20 broadcasting program display window. 

19. The interactive operation supporting system according 
to claim 17, wherein EPG distributed as part of data broadcast 
is applied for generating a list of broadcasting programs in 

25 a matrix form. 

20. The interactive operation supporting system according 
to claim 1, further comprising a connection means for 
connecting a television set/monitor to said system, a 

30 communication means for connecting said system to a 
communication medium such as an external network and/or a 



71 



telephone line, and a mail exchange means for making exchange 
of electronic mails via said communication medium, wherein 
said assistant control means makes an incoming mail display 
image to appear on a screen of said display unit in response 
5 to acceptance of mail. 

21. The interactive operation supporting system according 
to claim 1 , further comprising a text conversion means for 
converting a Japanese ideogram Kanji related to text data 

10 displayed on a screen of said display unit into a corresponding 
phonetic character Kana. 

22. The interactive operation supporting system according 
to claim 1 , further comprising a communication means for 

15 connecting said system to a communication medium such as an 
external network or a telephone line, and a certifying means 
for certifying an information terminal connected to said system 
via said communication medium. 

20 23. The interactive operation supporting system according 
to claim 1 , further comprising a connection means for connecting 
a television set /monitor to said system, and an extraction 
means for extracting text information from a received 
broadcasting program content. 

25 

24. The interactive operation supporting system according 
to claim 23, wherein text information extracted by said 
extraction means is superimposed on a content of another 
broadcasting program on said screen. 

30 

25. A method for supporting interactive operation of an 
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apparatus or other externally connected apparatus including 
a display unit , a speech input unit and a speech output unit , 
comprising : 

an assistant control step for generating a personified 
5 assistant andmaking said generated assistant appear on a screen 
of said display unit; 

an output speech control step for determining a speech 
required for the assistant to output the assistant's speech 
to the outside through said speech output unit after speech 
10 synthesis; 

an input speech recognition step for recognizing a user ' s 
voice as a speech inputted through said speech input unit; 

an interaction management step for managing interaction 
between said assistant and said user according to said 
15 assistant's speech determined by said output speech control 
step and said user speech recognized by said input speech 
recognition step; and 

a command interpreting step for specifying said user's 
intention or specifying said inputted user command based on 
20 a content of interaction traced by said interaction management 
step. 

26 . The method for supporting interactive operation 
according to claim 25, wherein said assistant control step 
25 determines an animation of said assistant based on a content 
of interaction managed by said interaction management step 
and/or an inputted user command specified by said command 
interpreting step. 

30 27 . The method for supporting interactive operation 
according to claim 25, wherein said output speech control step 
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determines an assistant's speech based on the content of 
interaction managed by said interaction management step and/ or 
an inputted user command specif ied by said command interpreting 
step. 

5 

28. The method for supporting interactive operation 
according to claim 25, wherein said output speech control step 
determines an assistant ' s speech for leading a user ' s intention 
when said command interpreting step fails to specify said user ' s 

10 intention or said inputted user command. 

29. The method for supporting interactive operation 
according to claim 25, wherein said apparatus further includes 
a connection means for connecting an external apparatus such 

15 as a television set/monitor and/or a video deck to said 
apparatus, and said command interpreting step also interprets 
commands for controlling a function of said external apparatus 
including selection of a broadcast program and/or 
recording/reproducing of said broadcast program. 

20 

30 . The method for supporting interactive operation 
according to claim 25 , wherein said apparatus further includes 
a communication means for connecting said apparatus to a 
communication medium such as an external network and/or a 

25 telephone line, and said input speech recognition step 
recognizes audio data received via said communication medium. 

31. The method for supporting interactive operation 
according to claim 25 , wherein said apparatus further includes 

30 a communication means for connecting said apparatus to a 
communication medium such as an external network and/or a 
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telephone line , and a mail exchange means for making an exchange 
of electronic mails via said communication medium, said output 
speech control step determining said assistant ' s speech based 
on a content of an incoming mail. 

5 

32. The method for supporting interactive operation 
according to claim 25, wherein said interaction management 
step manages an input speech of a user as a message bound 
for another user, and said output speech control step 

10 determines said assistant's speech based on said message. 

33. The method for supporting interactive operation 
according to claim 25, wherein said assistant control step 
places said personified assistant in a room scattered with 

15 various kinds of objects including a link to an information 
resource . 

34. The method for supporting interactive operation 
according to claim 25, wherein said assistant control step 

20 places said personified assistant in a room scattered with 
various kinds of objects including a link to an information 
resource , and in response an interest of said user in a recording 
medium including a link to a music content placed in said room, 
said command interpreting step interprets an inputted command 

25 as a command for playing back said music content. 

35. The method for supporting interactive operation 
according to claim 25, wherein in response to said command 
interpreting step interpreting an inputted user command, said 

30 assistant control step makes said assistant appear on a screen 
of said display unit . 



75 



36. The method for supporting interactive operation 
according to claim 25, wherein said apparatus further includes 
a connection means for connecting a television set/monitor 
5 to said apparatus , and in response to said command interpreting 
step interpreting an inputted user command as a channel select 
command, said assistant control step makes said assistant 
show a selected broadcasting program display window. 

10 37. The method for supporting interactive operation 
according to claim 25, wherein said apparatus further includes 
a connection means for connecting a television set/monitor 
to said apparatus , and in response to that said command 
interpreting step interpreting an inputted user command as 

15 a channel change command, said assistant control step places 
a changeable broadcasting program display window with a shape 
of a ring around said assistant. 

38. The method for supporting interactive operation 
20 according to claim 37, wherein, in response to selection of 

a desired channel by moving said display window on said ring 
upon a channel change command from said user and said assistant 
control step also zooms up said selected broadcasting program 
display window. 

25 

39. The method for supporting interactive operation 
according to claim 25 , wherein said apparatus further includes 
a connection means for connecting a secondary storage device 
for storing and/or reproducing a broadcasting program content 

30 to said apparatus , and in response to said command interpreting 
step interpreting an inputted user command as a recorded program 
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reproduction command, said assistant control step makes said 
assistant show a binder showing a view of recorded broadcasting 
program contents. 

5 40. The method for supporting interactive operation 
according to claim 39, wherein in response to a selection for 
reproducing a recorded broadcasting program content, said 
assistant control step zooms up said selected recorded 
broadcasting program content display window. 

10 

41. The method for supporting interactive operation 
according to claim 25 , wherein said apparatus further includes 
a connection means for connecting a television set/monitor 
to said apparatus , and in response to said command interpreting 
15 step interpreting an inputted user command as a channel change 
command, said assistant control step makes said assistant 
appear with a list of changeable broadcasting programs arranged 
in a matrix shape . 

20 42 . The method for supporting interactive operation 
according to claim 41, wherein in response to a selection of 
a desired channel, said assistant control step zooms up said 
selected broadcasting program display window. 

25 43. The method for supporting interactive operation 
according to claim 41, wherein EPG distributed as part of data 
broadcast is applied for generating a list of broadcasting 
programs in a matrix form. 

30 44. The method for supporting interactive operation 
according to claim 25 , wherein said apparatus further includes 
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a connection means for connecting a television set /monitor 
to said apparatus, a communication means for connecting said 
apparatus to a communication medium such as an external network 
and/ or a telephone line, and a mail exchange means for making 
5 exchange of electronic mails via said communication medium, 
and said assistant control step makes an incoming mail display 
image to appear on a screen of said display unit in response 
to acceptance of mail . 

10 45. The method for supporting interactive operation 
according to claim 25, further comprising a text conversion 
step for converting a Japanese ideogram Kanji related to text 
data displayed on a screen of said display unit into a 
corresponding phonetic character Kana. 

15 

46. The method for supporting interactive operation 
according to claim 25 , further comprising a communication step 
for connecting said apparatus to a communication medium such 
as an external network or a telephone line, and a certifying 

20 step for certifying an information terminal connected to said 
apparatus via said communication medium. 

47. The method for supporting interactive operation 
according to claim 25, wherein said apparatus further includes 

25 a connection means for connecting a television set/monitor 
to said apparatus, and an extraction step for extracting text 
information from a received broadcasting program content . 

48. The method for supporting interactive operation 
30 according to claim 47, wherein text information extracted by 

said extraction step is superimposed on a content of another 
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broadcasting program on said screen. 

49. A storage medium storing a software program in 
computer- readable form, said software program describing an 
5 interactive operation support processing for execution on a 
computer system, said interactive operation support 
processing applied to an apparatus including a display unit, 
a speech input unit and a speech output unit for supporting 
input of a user command to said apparatus and/or another 
10 externally connected apparatus , wherein said software program 
includes : 

an assistant control step for generating a personified 
assistant to make said generated assistant appear on a screen 
of said display unit : 
15 an output speech control step for determining a speech 

required to said assistant for output of said assistant ' s speech 
to the outside through said speech output unit after speech 
synthesis ; 

an input speech recognition step for recognizing auser ' s 
20 voice inputted through said speech input unit as speech; 

an interaction management step for managing interaction 
between said user and said assistant according to said 
assistant ' s speech determined by said output speech control 
step and said user speech recognized by said input speech 
25 recognition step; and 

a command interpreting step for specifying said user's 
intention or said inputted user command based on a content 
of interaction traced by said interaction management step. 

30 50. The interactive operation supporting system according 
to claim 1, further comprising a code conversion means for 
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converting a character related to text data according to a 
group or system of characters or codes displayed on said display 
unit into a corresponding character of another group or system 
of characters or codes . 

5 

51. The method for supporting interactive operation 
according to claim 25, further comprising a code conversion 
step for converting a character related to text data according 
to a group or system of characters or codes displayed on said 
10 display unit into a corresponding character of another group 
or system of characters or codes . 
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